(19) 




(12) 



Europaisches Patentamt 
European Patent Office 
Olfice europeen des brevets 



(43) Date of publication: 

12.11.1997 Bulletin 1997/46 



(n) EP 0 806 864 A2 

EUROPEAN PATENT APPLICATION 

(51) Intel e H04N 1/40 



(21) Application number: 97303137.0 

(22) Date of filing: 08.05.1997 



(84) Designated Contracting States: 


• Williams, Dawn M. 


DE FR GB 


Webster NY 14580 (US) 


(30) Priority: 08.05.1996 US 646865 


(74) Representative: Mackett, Margaret Dawn 




Rank Xerox Ltd 


(71) Applicant: XEROX CORPORATION 


Patent Department 


Rochester New York 14644 (US) 


Parkway 




Marlow Buckinghamshire SL7 1 YL (GB) 


(72) Inventors: 




• Schweld, Stuart A. 




Henrietta NY 14467 (US) 





(54) Image processing systems 



(57) Described herein is a filter which includes a plu- 
rality of one-dimensional first dimension filters (3, 7, 11 , 
15) operating in a first dimension, each of the filters op- 
erating on the digital image data in accordance with pre- 
determined filter coefficients to produce a one-dimen- 
sional output signal. The filter also includes a plurality 
of second one-dimensional filters (5. 9, 13, 17), operat- 
ing in a second, distinct dimension, each one-dimen- 



sional second dimension filter receiving output signals 
from one of the plurality of the first dimension filters and 
operating on the received output signals in accordance 
with predetermined filter coefficients to produce second 
filtered output signals therefrom. A weighting circuit (1 0, 
si, 12, s2, 14, s3, 16, SQ) is operatively connected to 
the one-dimensional second dimension filters to modify 
each of the produced second filtered output signals ac- 
cording to weighting coefficients. 
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Description 

The present invention relates generally to image processing systems, and is more particularly concerned with such 
systems which utilise digital tilters. 

5 In the reproduction of images from an original document or images from video image data, and more particularly 

to the rendering ot image data representing an original document that has been electronically scanned, one is faced 
with the limited resolution capabilities of the rendering system and the fact that output devices are mostly binary or 
require compression to binary for slorage efficiency. This is particularly evident when attempting to reproduce halftones, 
lines, and continuous tone (contone) images. 

io An image data processing system may be tailored so as to offset the limited resolution capabilities of the rendering 

apparatus, but this tailoring is difficult due to the divergent processing needs required by different types of images 
which may be encountered by the rendering device. In this respect, it should be understood that the image content of 
the original document may consist entirely of multiple image types, including high frequency halftones, low frequency 
halftones, continuous tones (contones), line copy, error diffused images, etc. or a combination of any of the above, 

*5 and some unknown degree of some or all of the above or additional image types. 

In view of the situation, optimizing the image processing system for one image type in an effort to offset the limi- 
tations in the resolution and the depth capability of the rendering apparatus may not be possible, requiring a compro- 
mised choice which may not produce acceptable results. Thus, for example, where one optimizes the system for low 
frequency halftones, it often at the expense of degraded rendering of high frequency halftones, or of line copy, and 

20 vice versa. 

To address this particular situation, 'prior art" devices have utilized automatic image segmentation to serve as a 
tool to identify different image types or imagery. For example, in one such system, image segmentation was addressed 
by applying a function instructing the image processing system as to the type of image data present, in particular, an 
auto-correlation function to the stream of pixel data to determine the existence of halftone image data. Such a method 

25 automatically processes a stream of image pixels representing unknown combinations of high and low frequency half- 
tones, contones, and/or lines. The auto-correlation function was applied to the stream of image pixels, and for the 
portions of the stream that contain high frequency halftone image data, the function produced a large number of closely 
spaced peaks in the resultant signal. 

In another auto segmentation process, an auto-correlatton function is calculated for the stream of halftone image 

30 data at selected time delays which are predicted to be indicative of the image frequency characteristics, without prior 
thresholding. Valleys in the resulting auto-correlated function are detected to determine whether high frequency halftone 
image is present. 

An example of a "prior art" automatic segmentation circuit is illustrated in Figure 15. The basic system as shown 
in Figure 15 is made up of three modules. Input information stored in a data buffer 10 is simultaneously directed to an 

35 image property classifying section 20, the first module, and an image processing section 30 : the second module, via 
databus 15. The image property classifying section 20 is made up of any number of submodules, (e.g. auto-correlator 
21 and discriminator 22), which determine whether a block of image pixels stored in the data buffer 10 is one type of 
imagery or another (e.g. halftone, line/text, or contone). In parallel with the image property classifying section 20, the 
image processing section 30 is made up of any number of sub-processing sections (e.g. high frequency halftone proc- 

40 essor 31 , low frequency halftone processor 32, lineAext processor 33, or contone processor 34), which perform image 
processing operations on the same block of image pixels as section 20. Each image sub-processing section performs 
image processing operations that are adapted to improve the image quality of a distinct class of imagery. The third 
module, control section 41, for example, an image processing mixer, uses the information derived from the image 
classifying section 20, to control the image processing section 30. 

45 The decision as to what class of imagery a block of image data belongs to is typically binary in nature. For example, 

in a conventional image segmentation scheme image property classifying section 20 classifies each pixel as one of 
three classes of imagery (high frequency halftone, low frequency halftone, or contone). Depending on those classifi- 
cation, each pixel is processed according to the properties of that class of imagery (either low pass filter and re-screen- 
ing if it is a high frequency halftone, threshold with a random threshold if it is a low frequency halftone, etc.). Also, 

so assuming that the decision as to which of the three classes of imagery a pixel belongs is based on a single image 
property, the peak count of the input image data, the resulting image classification decision of the peak count image 
property is made by thresholding the peak count into three classes of imagery as shown in Figure 16. 

Consequently, the control section 41 decides the type of image processing a block of image pixels requires de- 
pending on the decision made by the classification section 20. Thus, the output of classification section 20 is quantized 

55 to one of three possibilities. The control section 41 selects the output from one of the three image sub-processing 
sections based upon this classification. 

Image classification decisions using thresholds are usually artificially abrupt since an image can change from one 
class of imagery to another slowly and gradually. This abrupt decision making, which produces a forced choice among 
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several distinct alternative choices, is a primary reason for the formation of visible artifacts in the resulting output image. 
Most transition points or thresholds are selected so that an image can be classified as one class of imagery with a high 
degree of certainty; however, those classes of imagery that cannot be classified with such certainty have multiple 
transition points or a transition zone. Using, only one point to define a transition zone resutts in the formation of visible 

s artifacts in the resulting output image. Although it is possible to make the transition zone narrower so that there is less 
chance that an image falls into the zone, limitations exist on how narrow the zone can be made. 

In general, the "prior art" describes the control section 41 as essentially having a switch as illustrated in Figure 17. 
Since Ihe image processing steps performed for each class of imagery are different depending on the classification 
given to each block of input image pixels, the switch or multiplexer allows data residing at the output of the image 

10 processor 30 to be directed to an output buffer 50 depending on the decisions made by the imagery classifying section 
20 which are received as signals on lines 23 and 24. This type of binary decision making is rigid and results in image 
segmentation decisions that do not fail gracefully and consequently form visible artifacts in the output image. 

To address this forming of visible artifacts in the rendered output image, it has been proposed to utilize a probabilistic 
segmentation process to allow the image processing system to fail more gracefully when incorrect segmentation de- 

1S cisions are made. An example of such a probabilistic segmentation system is illustrated in Figure 11. 

Figure 11 shows a block diagram of an image processing system which incorporates a probabilistic classification 
system. As illustrated in Figure 1 1 , the system receives input image data derived from any number of sources, including 
a raster input scanner, a graphics workstation, an electronic memory, or other storage elements, etc. (not shown). In 
general, the image processing system shown in Figure 11 includes probabilistic classifier 25 having input control 18, 

20 image processing section 30, an image processing and control mixer 41 . . 

Input image data is made available to the image processing system along data bus 15, which is sequentially 
processed in parallel by probabilistic classifier 25 and image processing section 30. Probabilistic classifier 25 classifies 
the image data as a ratio of a number of predetermined classes of imagery. The ratio is defined by a set of probability 
values that predict the likelihood the image data is made up of a predetermined number of classes of imagery. The 

25 probabilities 27, one for each predetermined class of imagery, are input to the image processing mixer or control unit 
41 along with image output data from image processing section 30. 

Image processing section 30 includes units 31, 32, and 34 that generate output data from the image data in ac- 
cordance with methods unique to each predetermined class of imagery. Subsequently, mixer 41 combines a percentage 
of each class of output image data from units 31 , 32. and 34 according to the ratio of the probabilities 27 determined 

30 by classifier 25. The resulting output image data for mixer 41 is stored in output buffer 50 before subsequent transmis- 
sion to an image output terminal such as a printer or display. 

Initially, the stream of image pixels from an image input terminal (NT) is fed to data buffer 10. The image data 
stored in buffer 10 is in raw grey format, for example, 6 to 8 bits per pixel. A suitable block size is 16 pixels at 400 spots 
per inch, or 12 pixels at 300 spots per inch. Too large a sample size has a tendency to cause a blurred result, while 

35 too small a sample size does not contain a sufficient amount of data for a good sample. 

With reference to Figure 12, the probabilistic classifier 25 is shown in detail. The block of image pixels stored in 
buffer 10 is transmitted to a characteristic calculator 28 through data bus 15. Calculator 28 provides an output value 
that characterizes a property of the image data transmitted from buffer 10, such as its peak count. In one embodiment, 
a characteristic value is determined by calculator 28 that represents the peak count of the block of image data. The 

40 peak count is determined by counting those pixels whose values are the non-trivial local area maximum or minimum 
in the block of image data. First local area maximum or minimum pixel values are selected depending on whether the 
average value of all the pixels in the block of image data is higher or lower than the median value of the number of 
levels of each pixel. 

After calculator 28 evaluates the peak count of the image data, probability classifier 29 determines three probability 
45 values 27 that correspond to each image type associated with the peak count as expressed by the characteristic 
function stored in memory 26. The characteristic function, determined with a priori image data, represents a plurality 
of probability distributions that are determined using a population of images. Each probability distribution depicts the 
probability that a block of image data is a certain type given the occurrence of an image property, a peak count. For 
example, the characteristic function stored in memory 26 can be represented by the graph shown in Figure 13, which 
50 relates the probability distributions for a contone 1 , low frequency halftone 2, and high frequency halftone 3 to the 
occurrence of a particular image characteristic, which in this example is a peak count. The characteristic function stored 
in memory 26 can be adjusted using input control 18. Using control 18, the resulting output image stored in buffer 50 
can be altered by modifying the characteristic function representing the different classes of imagery evaluated by the 
image processing system 30. 

55 Subsequently, probability classifier 29 determines each probability value by evaluating the probability distribution 

of each image type represented by the characteristic function stored in memory 26. After determining the probability 
values : classifier 29 outputs these results to image processing mixer or control 41 . 

The image processing section of Figure 11 operates concurrently with the probabilistic classifier 25 on the image 
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data stored in buffer 10. Image processing section 30 includes a high frequency halftone processing unit 31, a low 
frequency halftone processing unit 32, and a contone processing unit 34. Each processing unit processes all image 
data in accordance with a particular image type. Each of the processing units 31, 32, and 34 generates output blocks 
of unquantized video data. 

s Image processing control 41 mixes the data output blocks to form a composite block of output image signals that 

is stored in output buffer 50. The manner in which the output blocks are mixed is characterized by a ratio defined by 
the probability determined by the probabilistic classifier 25. 

Figure 14 shows the image processing mixer 41 in detail. Mixer 41 multiplies the output blocks with the probability, 
using multipliers 42, 43, 44. The resulting output from each multiplier is representative of a percentage or ratio of each 

io output block, the sum of which defines a composite block of output image signals. The composite block of output image 
signals is formed by adding the output of the multipliers using adder 45 and by subsequently quantizing the sum of 
adder 45 using quantizer 47. The resulting image block output by quantizer 47 is stored in output buffer 50 before 
subsequent transmission for output to an image output terminal having limited resolution or depth. 

The above-described image classification system utilizes a probabilistic approach to classify the image data. Such 

is an approach presents problems in that the classification of the image data is mutually exclusive, the image data is 
classified as a particular type in absolute terms even though the probability of the decision being correct is just over 
50%. This results in difficulties in trying to design an image processing system which will process the image data without 
visible artifacts in the rendered image when the decision on the image type does not have a high confidence. 

Therefore, it is desirable to implement an image classification system which provides a truer classification of the 

20 image type and the image types are not necessarily mutually exclusive. Such a system could incorporate fuzzy logic, 
thereby allowing image data to be classified as being a member of more than one image class. This feature is critical 
in areas where the image goes from one image type to another. Moreover it is desirable to implement an image 
processing system which takes advantage of the fuzzy classification system. 

One important component of the rendering system is digital filtering. The digital filtering process should be both 

25 efficient and low cost. Moreover, the filter design should have some non-separable and/or time-varying characteristics 
so that the filter can be used in a fuzzy segmentation system. However, trying to achieve one goal or another can 
adversely impact the other goal. Various approaches have been devised for the implementation of digital filtering tech* 
niques which try to minimize the adverse impacts. These techniques will be discussed briefly below. 

In one "prior art" digital filtering technique, a circuit performs a plurality of finite impulse response filtering functions. 

30 The circuit comprises a plurality of filters, each implementing a predetermined digital filter algorithm. A storage circuit 
stores coefficients and data operands utilizing the predetermined algorithm. An arithmetic unit is coupled to the storage 
circuit for performing predetermined arithmetic operations with selected coefficients and data operands. A sequencing 
control device sequentially selects operands from the storage circuit for input to the arithmetic unit. 

In another "prior art" digital filtering technique, a digital signal processing apparatus provides high speed digital 

35 filtering. This apparatus includes at least two digital filters in parallel and a multiplexer for alternately outputting the 
outputs to the filters. In a third "prior art" device, a two-dimensional finite impulse response filter having a plurality of 
filter portions of essentially identical construction are arranged in a parallel configuration. A de-multiplexer separates 
an input data signal comprising consecutive digital words and supplies each digital word in sequence to a separate 
filter portion. Subsequently, a multiplexer, coupled to the output of the filter portions, selectively outputs the filtered 

40 data from each filter portion in a sequence corresponding to the order of separation of the input data, thereby resulting 
in a filtered version of the original input data. 

The three systems described above all have the limitation with respect to either speed or high cost. In view of 
these limitations, it has been proposed to provide a plurality of one-dimensional transform units that may be selectively 
combined with an additional one-dimensional transform unit to produce a plurality of distinct two-dimensional filters, 

45 any one of which is selectable on a pixel by pixel basis. Moreover, this proposed system has the added advantage of 
providing two-dimensional finite impulse response filters without employing multiple, identically constructed two-di- 
mensional filters arranged in a parallel fashion, thereby substantially reducing the complexity and cost of the filter 
hardware. To get a better understanding of this system, the system will be described below. 

The system, as illustrated in Figure 1, includes image processing module 20 which generally receives offset and 

so gain corrected video through input line 22. Subsequently, the image processing module 20 processes the input video 
data according to control signals from CPU 24 to produce output video signals. As illustrated in Figure 1, the image 
processing module 20 may include an optional segmentation block 30 which has an associated line buffer 32, two- 
dimensional filters 34, and an optional one-dimensional effects block 36. Also included in image processing module 
20 is line buffer memory 38 for storing the context of incoming scanlines 42A, 42B, 42C, 42D, 42E. 

55 Segmentation block 30, in conjunction with the associated scanline buffer 32, automatically determines those areas 

of the image which are representative of halftone input region. Output from the segmentation block (video class), is 
used to implement subsequent image processing effects in accordance with a type or class of video signals identified 
by the segmentation block. For example, the segmentation block may identify a region containing data representative 
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of an input halftone image, in which case a lowpass filter would be used to remove screen patterns, otherwise, a 
remaining text portion of the input video image may be processed with an edge enhancement filter to improve fine line 
and character reproduction when thresholded. 

Two-dimensional filter block 34 is intended to process the incoming, corrected video in accordance with the pre- 
s determined filtering selection. Prior to establishment of the required scanline content, the input video bypasses the 
filter by using a bypass channel within the two-dimensional filter hardware. This bypass is necessary to avoid deleterious 
effects to the video stream that may result from filtering of the input video prior to establishing the proper context. 

Subsequent to two-dimensional filtering, the optional one-dimensional effects block is used lo alter the filtered., or 
possibly unfiltered, video data in accordance with selected one-dimensional video effects. One-dimensional video ef- 
io fects include, for example, thresholding, screening, inversion, tonal reproduction curve (TRC), pixel masking, one- 
dimensional scaling, and other effects which may be applied one-dimensionally to the steam of video signals. As in 
the two-dimensional filter, the one-dimensional effects blocks also includes a bypass channel where no additional 
effects would be applied to the video, thereby enabling the received video to be passed through as an output video. 

Figure 2 illustrates the hardware of the two-dimensional filter of the device described above The hardware provides 
15 two independent finite impulse response (FIR), or convolution filters. Each filter allows for the use of a center pixel 
adjust coefficient although filter coefficients must be symmetric in both the fastscan and slowscan directions. The filter 
coefficients are programmable and may be set to zero to employ the use of a smaller filter. 

Each filter processes a predetermined number of input scanlines at a time pixel by pixel, to calculate each output 
scanline. As previously described with respect to Figure 1, input scanlines are buffered in line buffer 38 to meet the 
20 filter input requirements. In addition, if the two-dimensional filter 34 is used with segmentation block 30 of Figure 1, 
the filter must operate one scanline later than the segmentation block. 

As illustrated in Figure 2, input video is provided to slowscan filters 102A and 102B from line buffer memory 38 of 
Figure 1 . Also provided to filters 1 02A and 1 02B are the associated slowscan filter coefficient, as contained in slowscan 
filter coefficient memory 104A and 104B, respectively. The slowscan filter coefficients are arranged in a symmetric 
25 fashion about a center scanline coefficient. The output of slowscan filters 102A and 102B is directed to respective 
slowscan filter context buffers 106A and 106B. The output Irom the slowscan filter context buffers 106 A and 106B are 
fed to a multiplexer 208 which selects between which output is to be processed by a fastscan filter 230. 

The selection by the multiplexer is controlled by the class type of the video data being processed which was 
determined by the segmentation block 30 of Figure 1 . The fastscan filter coefficients to be utilized by the fastscan filter 
30 230 are received from one of a plurality of fastscan filter coefficient buffers wherein the context of the buffers are fed 
into a multiplexer 242 that selects the proper filter coefficients based on the class or image type of the video being 
processed. 

Although the above-described two-dimensional filter provides a filtering technique that is efficient, the filter is not 
time-varying as would be required if the image processing system utilized a fuzzy segmentation process. Therefore, 
35 it is desirable to design a digital filter that is non-separable and/or time-varying so that it can readily implemented in a 
fuzzy segmentation system. Moreover, it is desirable to design a time-varying filter that is self-timed and which is 
cascadable so as to enable proper filtering of the image data and provide scalability. 

It is therefore an object of the present invention to provide a filter which overcomes the problems of the prior art 
and which has flexibility of programming. Moreover, it is an object to provide a filter which can be utilized in a fuzzy 
40 segmentation system. 

In accordance with one aspect of the present invention, there is provided a filter device providing a plurality of two- 
dimensional digital filters. The filter device includes a plurality of one-dimensional first stage filters for producing a one- 
dimensional filtered output signal from an array of input pixel signals, a plurality of one-dimensional second stage filters, 
each one-dimensional second stage filter receiving output signals from a selected first stage filter and applying a second 
45 filtering operation thereto, to produce two-dimensional filtered output signals, and a weighting circuit to modify each of 
the produced two-dimensional filtered output signals according to weighting coefficients. 

In accordance with a second aspect of the present invention, there is provided a method for implementing a plurality 
of two<Jimensional digital filters. The method applies a first set of a plurality of one-dimensional first dimension filter 
to digital signals oriented along a first direction, each first dimension filters operating on the digital signals in accordance 
50 with a set of filter coefficient, applies a second set of a plurality of one-dimensional second dimension filters to the 
signals filtered by the one-dimensional first dimension filters oriented along a second direction in accordance with a 
set of filter coefficients, and weights each of the signals according to a set of weighting coefficients. 

One broad aspect of the present invention provides a non-separable filter having a plurality of separable filters to 
filter the digital signals to produce a plurality of filtered signals and a weighting circuit to weigh the plurality of filtered 
55 signals based on a set of weighting coefficients to produce a plurality of weighted signals 

Another broad aspect of the present invention provides a method for implementing non-separable filtering by fil- 
tering digital signals with a plurality of separable filter to produce a plurality of filtered signals, weighting the plurality 
of filtered signals based on a set of weighting coefficients to produce a plurality of weighted signals, and combining 
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the plurality of weighted signals to produce a single filtered signal. In one embodiment, the filtering is two<Jimensional 
and in another embodiment, the filtering is three-dimensional. 

The present invention also provides an image processing device for filtering digital image data. The device includes 
a plurality of one-dimensional first dimension filters operating in a first dimension, each of the filters operating on the 

5 digital image data in accordance with predetermined filter coefficients to produce a one-dimensionat output signal, and 
a plurality of one-dimensional filters, operating in a second, distinct dimension, each one-dimensional second dimen- 
sion filter receiving output signals from one of the plurality of the first dimension filters and operating on the received 
output signals in accordance with predetermined filter coefficients to produce second filtered output signals therefrom. 
The device also includes a weighting circuit operatively connected to the one-dimensional second dimension filters to 

10 modify each of the produced second filtered output signals according to weighting coefficients. 

The present invention also provides a video processing device for filtering a plurality of frames of video data. The 
device includes a plurality of one-dimensional first dimension filters operating in a first dimension, each of the first 
dimension filters filtering the video data based on a first set of filter coefficients; a plurality of one dimensional second 
dimension filters operating in a second dimension, each of the second dimension filters filtering the first dimensionally 

15 filtered video data based on a second set of filter coefficients; a plurality of one-dimensional third dimension filters 
operating in a third dimension, each of the third dimension filters filtering the second dimensionally filtered video data 
based on a third set of filter coefficients; and a weighting circuit to modify each of the produced third dimensionally 
filtered video data according to weighting coefficients. 

A method for implementing a non-separable three-dimensional filter by applying a plurality of first dimensional 

20 filters to video data is also provided in which the filtering is based on a first set of filter coefficients. A plurality of second 
dimensional filters are applied to the filtered video data and the filtering is based on a second set of filter coefficients. 
A plurality of third dimensional filters are applied to the second timed filtered video data and the filtering is based on a 
third set of filter coefficients. The third time filtered video data is weighted on the basis of a set of weighting coefficients. 
In another aspect of the present invention, an inseparable three-dimensional filter is provided that includes a plu- 

25 rality of separable three-dimensional filters to filter video signals to produce a plurality of filtered video signals; a weight- 
ing circuit to weigh the plurality of filtered video signals based on a set of weighting coefficients; and an adder to 
combine the plurality of weighted video signals to produce a single pixel of video data. 

In a further aspect of the present invention, there is provided a method for implementing an inseparable three- 
dimensional filters by applying a plurality of separable three-dimensional filters to a plurality of video signals to produce 

30 a plurality of filtered video signals; weighting the filtered video signals; and adding the weighted video signals to produce 
a single pixel of video data. 

For a better understanding of the present invention, reference will now be made, by way of example only, to the 
accompanying drawings in which:- 

35 Figure 1 is a schematic illustration of an image processing hardware module incorporating a two<Jimensional filter; 

Figure 2 is a block diagram illustrating the reduced hardware component set used in the image processing two- 
dimensional filter of Figure 1 ; 

Figure 3 is a block diagram illustrating a non-separable two-dimensional filter according to one embodiment of the 
present invention; 

40 Figure 4 is a block diagram illustrating an image processing system utilizing the two<iimensional filter of Figure 3; 

Figure 5 is a block diagram illustrating a self timed non-separable two-dimensional filter according to a second 
embodiment of the present invention; 

Figure 6 is a timing diagram illustrating the sequence of events carried out by the self timed non-separable two- 
dimensional filter of Figure 5; 

45 Figure 7 is a block diagram illustrating a cascadable filter according to another embodiment of the present invention; 

Figure 8 is a block diagram illustrating a filter; 

Figure 9 is a block diagram illustrating the formation of a single filter utilizing two cascadable filters; 
Figure 10 is a block diagram illustrating a cascadable fastscan filter; 

Figure 11 is a block diagram illustrating an image processing system incorporating probabilistic segmentation; 
so Figure 12 is a block diagram detailing the probabilistic segmentor shown in Figure 11 ; 

Figure 13 shows an example of a characteristic function of the image property, peak count; 
Figure 14 is a block diagram illustrating in detail the image processing mixer shown in Figure 11; 
Figure 15 is a block diagram illustrating a "prior art 0 image processing system: 

Figure 1 6 shows the thresholding technique used by the "prior art" to determine an appropriate image processing 
55 technique for images having different image types; 

Figure 17 is a block diagram illustrating in detail the "prior art" image processing control circuit shown in Figure 15; 
Figure 18 is a block diagram of a digital filtering system incorporating the fuzzy classification process of the present 
invention; 
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Figure 19 is a block diagram illustrating a screening system incorporating the fuzzy classification process of the 
present invention; 

Figure 20 is a block diagram illustrating a more detailed version of the system illustrated in Figure 19; and 
Figure 21 illustrates a block diagram showing a chip having a plurality of cascadable filters. 

5 

In this description, the terms "image data" or "pixels" in the form of video image signals, which may be either analog 
or digital voltage representations of an image, indicate a representation of an image provided from a suitable source. 
For example, the image signals may be obtained through line by line scanning of an image bearing the original by one 
or more photosensitive elements, such as a multiple photosite array of charge couple devices commonly referred to 
10 as CCDs. Line by line scanning of an image bearing the original for the duration of image data is well known. 

Image data may also be derived by a computer workstation program in accordance with document creation appli- 
cation software or from a data storage device. In content, the original video image signals may be composed entirely 
of a single image component such as lines, text, low frequency halftones, high frequency halftones, contones, or any 
combination thereof. 

15 The following description also includes references to slowscan and fastscan digital image data when discussing 

the directionality of two-dimensional filtering architecture. For purposes of clarification, fastscan data is intended to 
refer to individual pixels located in a succession along a raster of image information, while slowscan data refers to data 
derived from a common raster position across multiple rasters or scanlines. 

As an example, slowscan data would be used to describe signals captured from a plurality of elements along a 

20 linear photosensitive array as the array is moved relative to a document. On the other hand, fastscan data would refer 
to the sequential signals collected along the length of the linear photosensitive array during a single exposure period 
which is also commonly referred to as a raster of data. 

Moreover, in describing the present invention, it is assumed that the video signal has a value in a range between 
0 and 255. However, any range from the video signal can be utilized in conjunction with the present invention. Further- 

25 more, in the following description, the term "grey level" will be used to describe both black and white and color appli- 
cations. 

Furthermore, in describing the present invention, the term "pixel" will be utilized. This term may refer to an electrical 
(or optical, if fiber optics are used), signal which represents the physical measurable optical properties at a physical 
definable area on a receiving medium The receiving medium can be any tangible document, photoreceptor, or marking 
30 material transfer medium. 

Moreover, the term "pixel" may refer to an electrical (or optical, if fiber optics are used), signal which represents 
the physically measurable optical properties at a physically definable area on the display medium. A plurality of the 
physically definable areas for both situations represent the physically measurable optical properties of an entire physical 
image to be rendered by either a material marking device, electrically or magnetic marking device, or optical display 
35 device. 

Lastly, the term "pixel" may refer to an electrical (or optical, if fiber optics are used), signal which represents physical 
optical property data generated from a signal photosensor cell when scanning a physical image so as to convert the 
physical optical properties of the physical image to an electronic or electrical representation. In other words, in this 
situation, a pixel is an electrical (or optical), representation of the physical optical properties of a physical image meas- 

40 ured at a physical definable area on an optical sensor. 

Many of the documents produced today are compound documents in that the documents are composed of several 
different sub-images that are of different image types or image classes. Some of the common types are text, photos 
(contones), and halftones. One reason for the increased appearance of compound documents is the widespread use 
of commercially available word processing and desktop publishing software that is able to generate them. 

45 As is well known, different types of images require different processing in order to provide optimal image quality. 

Conventionally, to automatically choose the best processing for different areas of an image, each area is classified into 
one of several pre-defined classes to determine how to render that part of the image. This image type or image class 
information can then be used to determine the appropriate processing required to obtain a good rendition of the image 
when printing, to choose a method of image compression, to determine if optical character recognition would be useful, 

50 etc. 

However, as noted previously, the classification process should not be so crisp so as to avoid problems when the 
input image is not very similar to any of the classes, or the input images properties straddle the border between two 
classes. 

For example, if a particular action is taken based upon a single class identification because the classes are mutually 
55 exclusive, it may create undesirable results for a non-prototype image. This is seen when rendering images for printing 
on a xerographic printer. The classification of the image can cause output artifacts such as when a halftone image is 
classified as a contone image. 

Another type of problem is that adjacent areas of the image may be classified differently due to small variations 
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in the image. This is called class switching. If this information is used for image enhancement and printing, the output 
may have objectionable artifacts due to local variations. Examples of these objectionable artifacts are grainy image 
outputs. 

To eliminate the above described problems, an image classification system which utilizes a fuzzy membership into 
each category or class can be used. In other words, the classes in a fuzzy classification system are not mutually 
exclusive, thereby eliminating problems with class switching and also allowing those areas to have processing different 
than that of any of the other pre-defined classes; i.e., the output can choose between a continuum of possible image 
processing techniques. 

In standard classification techniques, each area has one class assigned to it. In the fuzzy implementation of the 
present invention, each area has a classification vector assigned to it. Every element of the classification vector has 
a membership value associated with each of the pre-defined prototype classes. 

Similar to the creation of crisp classifiers, a set of heuristic rules are used to determine the form of the classifier. 
The following is an example of how heuristic rules are used to create a fuzzy classifier, according to the present in- 
vention. 

For illustrative purposes, an example of a two class non-fuzzy system is discussed. In this example, the system 
only classifies a particular region as either contone (i.e., grey pictorial) or text. An image may be considered text if 
there are a lot of edges and most pixels are black or white. If this is not true, the picture is considered contone. 

In order to determine edges, a variable relating to the Laplacian transform ol the image data at every point (pixel) 
is used. Atypical implementation for this type of segmentation may be if the summation of the squares of the Laplacian 
transform at every pixel in the subblock is greater than a predetermined summation threshold and the sum of the 
percentage of pixels with grey value less than a black threshold value and the percentage of pixels with grey value 
greater than a white threshold value is greater than a predetermined bi-modal threshold: the image is text, else the 
image is contone. 

In this example, since the parameters are device dependent, tests, which are known to those skilled in the art, 
would be run to determine the values of all of the parameters: the percentage of pixels with grey value less than a 
black threshold value, the percentage of pixels with grey value greater than a white threshold value, the summation 
threshold, and the bi-modal threshold before executing the segmentation routine. Note that only one class can be 
chosen, either text or contone. 

In order to implement a fuzzy classifier, several modifications must be made to the above described heuristic rules. 
As described above, there exists a single rule defining only text. If the condition for text is "not true," contone is chosen. 
In a fuzzy system, contone must have its own rule, since the text membership rule is not an absolute truth, but a relative 
truth that the classification is true. 

Moreover, even providing a rule for contones will not satisfy the excluded middle law; therefore, a third "other" 
class must be added to satisfy the constraints of fuzzy logic. Without the "other" class, it would be possible for mem- 
bership of the image in all classes to be very small. Thus, the "other" class creates a lower bound of a half (0.5) for 
the minimum membership in any given class. A minimum magnitude for the maximum membership assures that all 
actions/decisions made using the relative membership values are not extremely sensitive to the class memberships, 
which they would be if membership in all classes was small, thereby making the fuzzy classification more robust. 

In one example of a fuzzy classification scheme, the pixel or unit of image data has a membership in each of the 
three classes; text, image and "other". In other words, the pixel is no longer considered to be an element of just one 
of the mutually exclusive classes. However, if the determination for one class reach absolute certainty; i.e., the mem- 
bership in a single class is 1 and the other classes is 0; the fuzzy system does generate values which would represent 
a crisp system. 

In view of this non-exclusivity characteristic of the pixel image membership, the membership of the pixel is repre- 
sented by a membership vector, Vj, whose entries correspond to the membership of the pixel (image element) in each 
of the classes. Note, typically, there are no constraints on this vector other than all of its elements must be greater than 
or equal to 0 and less than or equal to 1 . However, since the fuzzy classification rules have been setup with a third 
"other" class, at least one of the elements of the vector must be greater than or equal to 0.5 and less than or equal to 1 . 

Using the two class example above, the fuzzy classification rules would be the following. If the summation of the 
squares of the Laplacian transform at every pixel in the subblock is greater than a predetermined summation threshold 
and the sum of the percentage of pixels with grey value less than a black threshold value and the percentage of pixels 
with grey value greater than a white threshold value is greater than a predetermined bi-modal threshold, the pixel would 
be assigned a membership value for the "text" class which is the minimal value associated with each of the conditional 
statements. 

To better understand this concept, the following brief explanation of fuzzy logic will be provided. In fuzzy logic, 
unlike Boolean logic, the results of the conditional statements do not generate either absolute true or absolute false, 
but a value corresponding to the amount of the resulting statement which is true. This result is due to the fact that the 
conditional statements are also not absolute. 
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For example, in the above described text rule, from testing, it may be determined that the midpoint (predetermined 
target condition value) of the fuzzy Laplacian transform summation condition should be 50. The midpoint represents 
maximum uncertainty as to whether the value 50, in this example, is a member of the class Large Laplacian transform. 
Moreover, from testing, it is determined that, with absolute certainty, a pixel is member of the Large Laplacian transform 

s (membership equals 1 .0) if the summation is equal to or greater than 75 (predetermined absolute condition value) and 
It is determined that, with absolute certainty, a pixel is not a member of the Large Laplacian transform (membership 
equals 0.0) if the summation is equal to or less than 25 (predetermined absolute condition value). Fuzzy logic allows 
the classifier to assign 0.5 (conditional value) to a result where the summation is 50 and linearly extrapolate to the 
assigned values (conditional values) to 1 and 0.0 for the values 75 and 25 respectively; i.e. , value 55 would be assigned 

10 a membership value of 0.6. Note that these values are device dependent, and thus : the midpoint and the range need 
to be determined for each individual device. 

Furthermore, from testing, it may be determined that the midpoint of the class bi-modal should be 80; with absolute 
certainty, a pixel is in the membership if the percentage sum is equal to or greater than 90; and, with absolute certainty, 
a pixel is not in the membership if the percentage sum is equal to or less than 70. Fuzzy logic allows the classifier to 

15 assign 0.5 to a result where the sum value is 80 and linearly extrapolate to the assigned values to 1 and 0.0 for the 
values 90 and 70 respectively; i.e., value 85 would be assigned a membership value of 0.75. Note that these values 
are device dependent, and thus, the midpoint and the range needs to be determined for each individual device. 

To further explain the fuzzy technique, it is assumed that the membership values for each conditional statement 
are 0.5 and 0.33 respectively. In this scenario, the membership value for the pixel for the class text would be 0.33 

20 because fuzzy logic treats "anded" statements as determining the minimal value for all the conditions and assigning 
the minimal value to the membership value. 

Using the contone rule of if the summation of the squares of the Laplacian transform at every pixel in the subblock 
is less than a predetermined summation threshold and the sum of the percentage of pixels with grey value less than 
a black threshold value and the percentage of pixels with grey value greater than a white threshold value is less than 

25 a predetermined bi-modal threshold, the image is "contone" , each conditional statement will be discussed in fuzzy 
logic terms. For example, in the above described rule, from testing, it may be determined that the midpoint of the fuzzy 
Laplacian transform summation condition should be 50. Moreover, from testing, it is determined that, with absolute 
certainty, a pixel is in the membership if the summation is equal to or less than 25 and it is determined that, with absolute 
certainty, a pixel is not in the membership if the summation is equal to or greater than 75. Fuzzy logic allows the classifier 

30 to assign 0.5 to a result where the summation is 50 and linearly extrapolate to the assigned values to 1 and 0.0 for the 
values 25 and 75 respectively; i.e., value 55 would be assigned a membership value of 0.4 

Furthermore, from testing, it may be determined that the midpoint of the fuzzy bi-modal condition should be 80; 
with absolute certainty, a pixel is in the membership if the sum is equal to or less than 70; and, with absolute certainty, 
a pixel is not in the membership if the sum is equal to greater 90. Fuzzy logic allows the classifier to assign 0.5 to a 

35 result where the percentage value is 80 and linearly extrapolate to the assigned values to 1 and 0.0 for the values 70 
and 90 respectively; i.e., value 85 would be assigned a membership value of 0.25. 

To further explain the fuzzy technique, it is assumed that the membership values for each conditional statement 
are 0.75 and 0.8 respectively. In this scenario, the membership value for the pixel for the class text would be 0.75 
because fuzzy logic treats "anded" statements as determining the minimal value for all the conditions and assigning 

40 the minimal value to the membership value. 

Lastly, the fuzzy rules states that if image is neither "text" or "contone", the image is "other". This last rule, which 
defines the "other" class, can be represented mathematically as n other (image)=min(1-u, text (image), 1-|i con!one (image)) 
where \i x (Y) is the membership of Y in the class X. Note that if u- (e xt( ima 9 e )' and Mcontone(' ma 9 e ) are smaller than 0.5 
then H 0 ther(' ma 9 e ) w 'l' be greater than 0.5 (as stated earlier). In the example, given above, |a lext (image) is equal to 0.33 

45 and |-i contone (image) ' s equal to 0.75, thus, jj other (image) would be equal to 0.25, with the resulting membership vector 
being [0.33 0.75 0.25]. Note the element values of the vector need not add up to 1 . 

The predicate of each the rules described above is extended to a fuzzy truth instead of an absolute truth to provide 
the element value for the membership vector. Thus, in order to make the inequality "Y is > X" a fuzzy truth, a membership 
function is defined for "> X u . Similarly, a fuzzy membership rule can be defined for < X (very often, the membership in 

so (< X) is equal to not (> X): (1 - membership of (> X)). 

For simplicity in implementation, the membership in (> X) is defined as follows: 



55 
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The value ol AX determines the level of fuzzification of the class; if AX is extremely small, then the definition reduces 
to the crisp definition of greater than. It is further noted that although the fuzzification has been described as a linear 
relationship, the function describing the values between the end points and the mid point may be any type of function. 
Moreover, the midpoint could represent absolute certainty in the class and have a membership value of 1 and the 
ondpoints represent absolute certainty of non-membership such that the membership values would graphically form 
a triangle with the midpoint being the peak. 

Returning to the multiple "Ifs" in the above rules, the membership of image in the class text is equal to the fuzzy 
value of the predicate, 

u text (image) = min(u >S | pThreshold (Hp 2 ) , m >Bimoda | Threshold (White+Black)) where Zip 2 is the summation of the 
squares of the Laplacian transforms. 

To expand to the processing of images on a typical xerographic laser printer requires separating images into several 
classes; for example, white, black, edge, pictorial, low frequency halftone, mid frequency halftone, high frequency 
halftone and other, etc. The classes white, black, and pictorial are subclasses of the set "contone" and low frequency 
halftone, mid frequency halftone, high frequency halftone are subclasses of the set "halftone'. 

In a preferred embodiment of the present invention, the deterministic values for determining membership are as 
follows: 

BLACK_% = the percentage of pixels with grey value less than a black threshold value; 

WHITE_% = the percentage of pixels with grey value greater than a white threshold value; 

Sij = Sum of the absolute values of the Laplacian transforms in a window around the pixel being classified; 

Range = Max grey Level - Min grey level inside a window around the pixel being classified; and 

Freq = Measurement of local 2-D frequency around the pixel being classified. 



To determine the membership value in a particular class, these values are compared to a variety of predetermined 
thresholds in a similar manner as described above with respect to the three class system. The various classes in this 
preferred embodiment are demonstrated by the following rules: 

35 

If (Sij is > SI J_H ALFTON E and RANGE is >RANG E_H ALFTONE and FREQ is >FREQJHALFTONE), then C1 is 
HALFTONE; 

If (Sij is > SIJ_EDGE and RANGE is > RANGE_E DGE and FREQ is < FREQ_H ALFTONE), then pixel is EDGE; 
If (Sij is <SIJ_HALFTONE and FREQ is <FREQ_HALFTONE), then C1 is CONTONE; 
40 |f (C1 is CONTONE and BLACK_% is >BLACK_THRESHOLD), then pixel is BLACK: 

If (C1 is CONTONE and WHITE_% is >WHITE_THRESHOLD), then pixel is WHITE: 

If <C1 is CONTONE and BLACK_% is < BLACK_TH RE S HOLD and WHITE_% is WHITE_THRESHOLD), then 
pixel is PICTORIAL; 

If (C1 is HALFTONE and FREQ is <LOW_FREQ), then pixel is LOW_FREQ_HALFTONE: 
45 |f (C1 is HALFTONE and FREQ is >LOW_FREQ and FREQ is <HIGH_FREQ), then pixel is 

MID_FREQ_HALFTONE; and 

If (C1 is HALFTONE and FREQ is>HIGH_FREQ), then pixel is HIGH_FREQJH ALFTONE: and 
If (pixel is not BLACK and pixel is not WHITE and pixel is not PICTORIAL and pixel is not EDGE and pixel is not 
LOW FREQ_HALFTONE and pixel is NOT MID_FREQ_H ALFTONE and pixel is not HIGH FREQ HALFTONE), 
so then pixel is OTHER. 

The predicate of each the rules described above is extended to a fuzzy truth instead of an absolute truth to provide 
the element value for the membership vector. Thus, in order to make the inequality n Y is > X" a fuzzy truth, a membership 
function is defined for M > X" Similarly, a fuzzy membership rule can be defined for < X (very often, the membership in 
55 (< X) is equal to not (> X): (1 - membership of (> X)). 

For simplicity in implementation, the membership in (> X) is again defined as follows: 
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10 The value of AX determines the level of f uzzif ication of the class; if AX is extremely small, then the definition reduces 

to the crisp definition of greater than. It is further noted that although the fuzzification has been described as a linear 
relationship, the function describing the values between the end points and the mid point may be any type of function. 
Moreover, the midpoint could represent absolute certainty in the class and have a membership value of 1 and the 
endpoints represent absolute certainty of non-membership such that the membership values would graphically form 

is a triangle with the midpoint being the peak. This type of class could be used for a membership function of the class "=X tt . 

Returning to the multiple "Its" in the above rules, the membership value of image in the class "edge" would be 
equal to the fuzzy value of the predicate, u edge (image) = min(m, slp Threshold (lip), u >Range Threshold (Max grey - Min grey ), 
M^-Freq Threshold 21 -* ^req)), the membership value of image in the class "black" would be equal to the fuzzy value of the 
predicate, u. black (image) = min(u. <s , pThresh0 | d (Zip), u <Freq Thre3hold (2D Freq), M>«aek Th«.how(% of black Pixels)); the 

20 membership value of image in the class "white" would be equal to the fuzzy value of the predicate, m^j^ image) = min 

(U <Slp Threshold 

(Lip), M<Freq Thre 6 hoid(2D Freq), u. <white Threehoid( 0 /° oi white Pixels)); the membership value of image in 
the class "pictorial" would be equal to the fuzzy value of the predicate, |i P ictoriai(' ma 9 e ) = minfu^sip Threshold (Sip), 

M.Freq Threshold(2D Freq), U. 

<Biack Threshold^ of black pixels), u,^ white Threshold^ of white pixels)); the membership value 
of image in the class "low frequency halftone" would be equal to the fuzzy value of the predicate, m owfreqhalf ( image) = 

25 min(U- >slp Threshold ( S, P)- u >Range Threshold* Max grey " Min grey)- M^Freq Threshold(2D Freq), U, <LowFreq T hreshold( 2 D Freq)); the 

membership value of image in the class "mid frequency halftone" would be equal to the fuzzy value of the predicate, 

u midfreqhalf( ima 9 e ) = mln (^>Slp Threshold ( £I P). ^> Range Thresh old ( Max grey " Min grey)- Freq Threshold* 2D Freq). 

^>LowFreqThreshoid(2D Freq), |i< High Freq Threshoid(2D Freq)); and the membership value of image in the class "high fre- 
quency halftone" would be equal to the fuzzy value of the predicate, u-highfreqhaif(' ma 9 e ) = min (l J >sip Threshold (^P)« 

30 M^Range Threshold( Max grey " Min grey)- M^>Freq Threshold( 2 D Freq), H >H jghFreq Threshold* 20 Fre q)) 

To implement the fuzzy segmentation process for the two image class situation, image data received from the 
image source is divided into blocks of pixels. The fuzzy image classifier then, to determine a membership value for a 
particular image type, calculates the summation of the squares of the Laplacian transforms in a window around the 
pixel being classified. Moreover, the fuzzy classifier calculates the percentage of pixels that have a grey value less 

35 than a predetermined black value and determines the percentage of pixels within the block which have a grey value 
greater than a predetermined white value. 

After calculating this information, the fuzzy classifier determines the conditional value for the condition relating to 
the summation of the squares of the Laplacian transform of every pixel in the block being greater than a Laplacian 
transform threshold and the conditional value for the condition relating to the sum of the percentage of pixels with a 

<to grey value greater than the predetermined black value and the percentage of the pixels with a grey value greater than 
the predetermined white value is greater than a bi-modal threshold value. Upon determining these two conditional 
values, the fuzzy classifier determines the minimal value and assigns this value as the membership value for the pixel 
in the "text" class. 

The fuzzy classifier then determines the conditional value for the condition relating to the summation of the squares 
45 of the Laplacian transform of every pixel in the block is less than a Laplacian transform threshold value and the con- 
ditional value for the condition relating to the summation of the percentage of the pixels with a grey value less than a 
predetermined black value and the percentage of the pixels with a grey value greater than the predetermined white 
value is less the bi-modal threshold value. Upon determining these two conditional values, the fuzzy classifier deter- 
mines the minimal value and assigns this value as the membership value for the pixel in the "contone" class. 
50 The fuzzy classifier thereafter determines the membership value for the pixel in the "other" class by determining 

the minimal value between 1-the "text" membership value and 1-the "contone" membership value and assigns this 
value as the membership value for the pixel in the "other" class. 

In this process, the pixels of image data received by the fuzzy classifier are assigned membership values for the 
three possible fuzzy classifications, "text", "contone" : or "other". As noted above, the utilization of the "other" class is 
55 necessary in order to avoid having membership of the image in all classes to be very small. 

An example of implementing fuzzy classification for a laser xerographic printer, will now be discussed, more spe- 
cifically, a process, carried out by a fuzzy classifier, to assign membership values to the pixels of image data for eight 
possible types or classes. 
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As with the process described above, the process begins by dividing the pixels of image data into blocks of pixels. 
Thereafter, each block of pixels is analyzed to determine the sum of the absolute values of the Laplacian transforms 
in a predetermined window around the pixel being presently analyzed; to determine a range value which is equal to 
the maximum grey level minus the minimum grey level inside the predetermined window around the pixel being pres- 

5 ently analyzed: to determine a frequency value which is equal to the measurement of the local two-dimensional fre- 
quency around the pixel being presently analyzed; to determine a black value which is equal to the percentage of pixels 
that have a grey value less than a predetermined black value; and to determine a white value which is equal to the 
percentage of pixels wilhin the window having a grey value greater than a predetermined white value. 

Once these various values are determined, the fuzzy classifier begins the assigning of the membership values. 

10 The fuzzy classifier determines the conditional value for the condition relating to the sum of the absolute values of the 
Laplacian transform in the predetermined window around the pixel being presently analyzed being greater than a 
halftone threshold, the conditional value for the condition relating to the range value being greater than a range halftone 
threshold, and the conditional value for the condition relating to the frequency value being greater than a frequency 
halftone threshold. Upon determining these three conditional values, the fuzzy classifier determines the minimal value 

15 and assigns this value as the membership value for the pixel in the "halftone" class. 

Then, the fuzzy classifier determines the conditional value for the condition relating to the summation of the absolute 
values of the Laplacian transforms in the predetermined window around the pixel being presently analyzed being 
greater than an edge threshold, the conditional value for the condition relating to the range value being greater than 
a range edge threshold, and the conditional value for the condition relating to the frequency value being less than the 

20 frequency halftone threshold. Upon determining these three conditional values, the fuzzy classifier determines the 
minimal value and assigns this value as the membership value for the pixel in the "edge" class. 

The fuzzy classifier thereafter determines the conditional value for the condition relating to the sum of the absolute 
values of the Laplacian transforms in the predetermined window around the pixel being presently analyzed being less 
than the halftone threshold and the conditional value for the condition relating to the frequency value being less than 

25 the frequency halftone threshold. Upon determining these two conditional values, the fuzzy classifier determines the 
minimal value and assigns this value as the membership value for the pixel in the "contone" class. 

The fuzzy classifier then determines the conditional value for the condition relating to the pixel being a contone 
image and the conditional value for the condition relating to the black value being greater than a black threshold value. 
Upon determining these two conditional values, the fuzzy classifier determines the minimal value and assigns this 

30 value as the membership value for the pixel in the "black" class. 

Subsequently, the fuzzy classifier determines the conditional value for the condition relating to the pixel being a 
contone image and the conditional value for the condition relating to the white value being greater than the predeter- 
mined white threshold. Upon determining these two conditional values, the fuzzy classifier determines the minimal 
value and assigns this value as the membership value for the pixel in the "white" class. 

35 The fuzzy classifier determines the conditional value for the condition relating to the pixel being a halftone image 

and the conditional value for the condition relating to the frequency value being less than a low frequency threshold 
value. Upon determining these two conditional values, the fuzzy classifier determines the minimal value and assigns 
this value as the membership value for the pixel in the "low frequency halltone" class. Thereafter, the fuzzy classifier 
determines the conditional value for the condition relating to the pixel being a halftone image, the conditional value for 

40 the condition relating to the frequency value being greater than the low frequency threshold value, and the conditional 
value for the condition relating to the frequency value being less than a high frequency threshold value. Upon deter- 
mining these three conditional values, the fuzzy classifier determines the minimal value and assigns this value as the 
membership value for the pixel in the "mid frequency halftone" class. 

The fuzzy classifier determines the conditional value for the condition relating to the pixel being a halftone image 

45 and the conditional value for the condition relating to the frequency value being greater than the high frequency thresh- 
old value. Upon determining these two conditional values, the fuzzy classifier determines the minimal value and assigns 
this value as the membership value for the pixel in the "high frequency halftone" class. 

Lastly, the fuzzy classifier determines the membership value for the pixel in the "other" class by determining the 
minimal value between 1-the "edge" membership value, 1-the "black" membership value, 1-the "white" membership 

50 value, 1-the "pictorial" membership value, 1-the "low frequency halftone" membership value, 1-the "mid frequency 
halftone" membership value, and 1-the "high frequency halftone" membership value and assigns this value as the 
membership value for the pixel in the "other" class. 

By utilizing these processes, the fuzzy image classifier can eliminate problems with class switching in areas be- 
tween two or more pre-defined types. In other words, these processes implement a fuzzy classification scheme which 

55 allows the defining of various image types or classes to fail gracefully Moreover, by implementing a luzzy classification 
process, the fuzzy process allows the fuzzy memberships to have enhancement different than any of the pre-defined 
classes. More specifically, the image processing system can choose between a continuum of possible processing 
operations to produce an output. 
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The described embodiment also allows the determination of output parameters such as filter coefficients and 
screening level given a fuzzy classification vector and a corresponding desired output for each prototype class. The 
described embodiment can also accommodate output types that must satisfy certain constraints (such as the mean 
gain of the filter) through the use of the "other" class. The fuzzy image processing greatly attenuates switching noise 

5 common in crisp decision image segmentation processes. 

Processing image data for laser printer reproduction requires the implementation of two separate actions: image 
processing to enhance, through manipulation, which does not result in loss of information, such as filtering and TRC 
translation; and a lossy conversion of the resultant image to a representation accepted by a print engine, normally a 
reduction in the number of bits per pixel through either application of a screen or error diffusion. By utilizing a fuzzy 

10 classification scheme, the described embodiment can easily unify the processing implied by all of the partial member- 
ships into a single processing action. 

For example, each of the classes, defined using fuzzy classification, is provided with an ideal processing scheme. 
This ideal image processing scheme would represent the image processing techniques used if the membership in that 
class was one (1.0) and all other classes had a membership of zero (0.0). The fuzzy image classification process, 

is described above, provides the mechanism for determining the output for given fuzzy membership because the output 
is a vector. Thus, fuzzy classification can be used to choose each element of the vector independently, thereby, as 
noted above, providing a continuum of possible image processing operations. However, this continuum is not directly 
conducive to determining screening operations and determining filter coefficients. More specifically, a screening oper- 
ation, by its nature, is discrete; either a screen is applied or it is not. At every point, a comparison is made between 

20 the image data (video level) and the. screen value (screen level) to determine if a pixel is to be turned OFF or ON. 
Thus, on its face, screening would appear to be a poor candidate for fuzzy classification control because of its inability 
to be applied at differing levels. 

The other problem arises with determining filter coefficients. Image processing requires that the filter coefficients 
sum to 1 (i.e., a constant grey level in produces the same grey level out). If there is the choice between several filters, 

25 their combination may no longer meet this requirement. 

Another embodiment resolves the first situation, the problem with applying a screen in a fuzzy classification envi- 
ronment, by replacing the screening operation with a linear operation that provides the same effect. Instead of com- 
paring all video to a pre-defined screen, a 2-D sinusoidal type function, with a maximum amplitude at 45 degrees, is 
added to the incoming video. A more detailed explanation of this type of screening is disclosed in EP-A-0 696 1 31. 

30 This screen, referred to as hybrid screening, creates video that is more likely to cluster output pixels in an error 

diffusion process. This clustering of video is the effect that is very pleasing to the human eye in constant or near constant 
grey areas. It is the clustering effect that makes this screened output superior to a straight non-screened error diffused 
output. 

The frequency characteristics of the 2-D hybrid screen determines the approximate line pair per inch (Ipi) dot 
35 pattern seen in the output; however, the ability to collect pixels into large dots, the desired effect in a screening process, 
is dependent on the screen's amplitude. This amplitude, a scalar, can easily be modified using the fuzzy image clas- 
sification process of the present invention. All of the rules can assign a level (i.e., amplitude) of screening (large, 
medium, small or none, etc.). The size and frequency of the screen are predetermined to match the characteristics of 
the printer. 

4 ° The image processing of an image using a fuzzy classification process is very similar to the fuzzy classification 

process itself. For example, in the screening case, the fuzzy image processing method would establish three screening 
classes (large, medium, and none) to determine the screen value to be applied to pixel Each of these classes would 
have a set of rules as in the fuzzy classification method to determine the membership value for the pixel in the screening 
processing class. The membership value would then be used to calculate the actual screen amplitude which will be 

45 described in more detail below. 

In a preferred embodiment of the present invention, the rules for the screening classes are as follows: 

If (pixel is "edge" or pixel is "low frequency halftone" or pixel is "mid frequency halftone" or pixel is "other"), then 
screen is NO_SCREEN; 
so |f (pixel is "black" or pixel is "white"), then screen is MEDIUM_SCREEN; or 

If (pixel is "pictorial" or pixel is "high frequency halftone"), then screen is FULL_SCREEN. 

Referring to the multiple "Ifs" in the above rules, the membership value of image in the class "NO_SCREEN" would 
be equal to the fuzzy value of the predicate, M NO _scREEN( screen ) = max(u. edge (pixel), u lowfreqhalftone (pixel), 

^midfreqhalftone 

55 (pixel), u. other (pixel)); the membership value of image in the class M MEDIUM_SCREEN" would be equal to the fuzzy 
value of the predicate, MMEDiUM_scREEN( screen ) = max (^btack(P ixe, )> m/vhite(P ixel )); and tne membership value of image 
in the class "FULL_SCREEN" would be equal to the fuzzy value of the predicate, HFUU__scREEN( screen ) = max (M P ictoriai 
(pixel), u hjghfreqhalftone (pixel)). " 
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To implement the fuzzy screening, the processing system determines the membership value of the pixel in each 
of the classes associated with a particular screening process and assigns the maximum value as the membership 
value in the screening class. For example, if the pixel had membership values for "edge", "low frequency halftone", 
"mid frequency halftone", and "other" of 0.6. 0.7, 0.2, and 0.3 respectively, the processing system would decode the 

s membership vector and assign a membership value to the NO_SCREEN class of 0.7. Moreover, if the pixel had mem- 
bership values for "black" and "white" of 0.6 and 0.5, respectively, the processing system would decode the membership 
vector and assign a membership value to the MEDIUM_SCREEN class of 0.6. Lastly, if the pixel had membership 
values for "pictorial" and "high frequency halftone" of 0.3 and 0.4 respectively, the processing system would decode 
the membership vector and assign a membership value to the FULL_SCREEN class of 0.4. 

10 To determine the actual amplitude for the screen value, the fuzzy processing module, in the preferred embodiment 

of the present invention, multiply the membership value for each screen class with the ideal coefficient for that class 
and divide the product by the sum of the membership values. In the preferred embodiment, the NO_SCREEN coefficient 
is 0, MEDIUM_SCREEN coefficient is 0.5, and FULL_SCREEN coefficient is 1.0. Thus, in the example described 
above, the screen amplitude for the pixel being processed would be equal to a scalar value of 0.41 2 (({1 .0*0.4)+(0.5*0.6) 

is +(0.0*0.7))/(0.4+0.6+0.7)). 

An example of the application of fuzzy classification to screening is illustrated in Figure 19. As illustrated in Figure 
19, the video signal or image data is fed into a fuzzy classifier 80 which classifies the image data according to the rules 
described above. The fuzzy image classifier 80 then generates a membership vector which is passed onto a screening 
generating circuit 88. The screen generating circuit 88 produces a screen value which is added to the image data at 

20 adder 89. The image data which is summed with the screen value corresponds to the same pixel as was classified by 
the fuzzy image classifier 80. In other words, the system illustrated in Figure 19 also includes buffers (not shown) to 
insure that the pixel being processed corresponds to the correct membership vector being used to define the processing 
parameters. 

A more detailed example of the application of fuzzy segmentation to screening is illustrated in Figure 20. As illus- 

25 trated in Figure 20, the video signal or image data is fed into a fuzzy classifier 80 which classifies the image data 
according to the rules described above. The fuzzy image classifier 80 then generates a membership vector which is 
passed onto a screening generating circuit 88. The screen generating circuit 88 includes a screening weighting circuit 
883, a screen value generating circuit 881 , and a multiplier 885. 

The screen weighting circuit 883 generates a weighting factor in response to the values in the membership vector 

30 so as to produce a no screen, a medium screen, or a high screen, or any screen therebetween, as discussed above. 
In other words, if the pixel had membership values for "edge", "low frequency halftone", "mid frequency halftone", and 
"other" of 0.8, 0.7. 0.1 , and 0.3 respectively, the processing system would decode the membership vector and assign 
a membership value to the NO_SCREEN class of 0.8. Moreover, if the pixel had membership values for "black" and 
"white" of 0.4 and 0.1 respectively, the processing system would decode the membership vector and assign a mem- 

35 bership value to the MEDIUM_SCREEN class of 0.4 Lastly, if the pixel had membership values for "pictorial" and "high 
frequency halftone" of 0.2 and 0.0 respectively, the processing system would decode the membership vector and assign 
a membership value to the FULL_SCREEN class of 0.2. 

Thus, in this example, the screen amplitude for the pixel being processed would be equal to a scalar value of 0.286 
(((1.0*0.2)+(0.5*0.4)+(0.0*0.8))/(0.2+0.4+0.8)). 

40 The screen value generating circuit 881, which may be a lookup table or a hardwired circuit, produces a screen 

value based on the position of the pixel (image data) within the image. The weighting factor from the screen weighting 
circuit 883 and the screen value from screen value generating circuit 881 are multiplied together by multiplier 885 to 
produce the screen value to be added to the pixel. This screen value is added to the image data at adder 89. 

The image data which is summed with the screen value corresponds to the same pixel as was classified by the 

4S fuzzy image classifier 80. In other words, the system illustrated in Figure 20 also includes buffers (not shown) to insure 
that the pixel being processed corresponds to the correct membership vector being used to define the processing 
parameters. 

As noted above, the problem of using a fuzzy image classification system to control digital filtering is two-fold. 
First, digital filtering is not a scalar function, but a matrix function. Secondly, there is a constraint on the matrix function; 
so the filter must have gain of 1 .0 at (co-,a)2)=(0,0). This ensures that constant grey areas are not altered by filtering. 

To solve this problem, the preferred embodiment uses the weighted summation of several pre-defined filters to 
produced the filtered results. These filters are filters associated with a particular filter class; i.e., one filter class is 
enhance, one filter class is lowpass, and another filter class is "other". A digital filter takes the form of F c = IpjFj where 
all of the FjS correspond to the filters associated with each i th class (or classes) and jij corresponds to the membership 
55 of the image in the ith class, as determined by the fuzzy processing routine. 

In a preferred embodiment of the present invention, the rules for the filtering classes are as follows; 

If (pixel is "edge" or pixel is "pictorial" or pixel is "low frequency halftone"), then filter is ENHANCE; or 
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If (pixel is "high frequency halftone"), then filter is LOWPASS. 

Referring to the multiple "Ifs" in the above rules, the membership value of image in the class "ENHANCE" would 
be equal to the fuzzy value of the predicate, u. ENHANCE (tilter) = max(u edge (pixel), UH 0wfreqhalftone (pixel), u pictorial (pixel)); 
s and the membership value of image in the class "LOWPASS" would be equal to the fuzzy value of the predicate, 

MLOWPASs( fi 'te r > = max(M highfreq haiftone(P ix el))- 

To implement the fuzzy screening, the processing system determines the membership value of the pixel in each 
of the classes associated with a particular screening process and assigns the maximum value as the membership 
value in the screening class. For example, if the pixel had membership values for "edge", "low frequency halftone", 
10 and "pictorial" of 06, 07, and 0.3 respectively, the processing system would decode the membership vector and assign 
a membership value to the ENHANCE class of 0.7. Moreover, if the pixel had membership values for "high frequency 
halftone" of 0.6, the processing system would decode the membership vector and assign a membership value to the 
LOWPASS class of 0.6. 

To determine the actual coefficients for the various filters, the fuzzy processing system must ensure that the fuzzy 
15 filtering, resulting from the rule set, meets the constraint of a gain of 1 .0 at the frequency (o^co^^O.O). To alleviate this 
problem, one of the output filter choices is assigned the bypass function. In bypass, Vout=Vin; i.e., no filtering is done. 
Thus, the resulting filter is F Q = Z^iJF t + (1 -£fjj) * F B such that when the desired effect is the bypass filter, the value goes 
to zero and the effect of the filter Fj is ignored. 

It is noted that the enhancement filter amplifies all higher frequencies, and the lowpass filter attenuates the higher 
20 frequencies. The coefficient value, c, for the bypass filter can be determined using c^^ = 1 - u. cn hanco( fj,ter ) * Miowpaco 
(filter) so that the output filter can be described as F 0 = n enhance (fiter) * F enhance + M lowpasi8 (filtGr) * F (owpaee + c bypa6C * 

^bypass- 

An example of image processing an image using fuzzy classification with respect to filtering the image data is 
illustrated in Figure 18. As illustrated in Figure 18, video data or image data is fed into a fuzzy classifier 80, a lowpass 
25 filter 81 , an enhanced filter 82, and a bypass filter 83 in parallel. As described above, the fuzzy classifier 80 determines 
the membership vector of the pixel to be processed by the parallel set of filters. Note that this process includes buffers 
(not shown) to insure that the pixel being filtered corresponds to the correct membership vector being used to define 
the processing parameters. 

Based upon this classification, the fuzzy classifier 80 will cause the overall filter to generate a set of coefficients 

30 which are applied to multipliers 84, 85, and 86. The coefficients will enable the overall filter to weight the output from 
the various filters according to the fuzzy image classification. 

For example, as noted above, if the pixel had membership values for "edge", "low frequency halftone", and "pic- 
torial" of 0.6, 0.7, and 0.3 respectively, the processing system would decode the membership vector and assign a 
membership value to the ENHANCE class of 0.7, which in turn is the filter coefficient assigned to the enhance filter 82 

35 and fed to multiplier 85. Moreover, it the pixel had membership values for "high frequency halftone" of 0.6 : the processing 
system would decode the membership vector and assign a membership value to the LOWPASS class of 0.6, which in 
turn is the filter coefficient assigned to the lowpass filter 81 and fed to multiplier 84. This leaves the generation of the 
coefficient for the bypass filter 83. 

As noted above, the generated coefficients need a relationship such that the sum of the coefficients will not be 

io greater than 1 so as to keep the gain of the overall output from the filters to be equal to 1. Thus, in the preferred 
embodiment of the present invention, the coefficient for the bypass filter is 1 minus the enhance coefficient minus the 
lowpass coefficient (in the example, 1 - 0.7 - 0.6= -0.3). This coefficient is applied to the multiplier 86. The weighted 
filter outputs are then fed into an adder 87 which adds all the outputs to produce a filtered pixel of image data which 
has been filtered according to its fuzzy image classification. 

45 Although Figure 18 illustrates the utilization of a fuzzy classifier with a plurality of different filters, the fuzzy classifier 

can also be utilized in connection with a modular time variant two-dimensional non-separable filter wherein the non- 
separable filter is made up of a plurality of separable filters. 

The utilization of separable filters allows a non-separable filter of a particular form to be implemented in a cost 
effective manner. Moreover, a separable two-dimensional filter can be described as one-dimensional filter that acts in 

so the vertical direction or slowscan direction followed by another one-dimensional filter acting in the horizontal or fastscan 
direction. Thus, the filter matrix can be described as the product of F vh = f v *f h wherein F vh is an N by M matrix, f v is a 
N length column vector (the vertical filter) and f h is a M length row vector (the horizontal filter). 

If the matrix F vh cannot be represented using the above equation, the matrix is considered non-separable and the 
separable filter implementation cannot be used. However, if the N by M matrix is represented using a singular value 

55 decomposition such as F vh = U*S*V, where U is N by N unitary matrix, S is an N by M diagonal matrix, and V is a M 
by M unitary matrix: a separable filter implementation can be used. Furthermore, if N and M are not equal, the above 
equation can be altered to F vh = U r *S r *V p where Q = min(n,m), U r is a N by X submatrix of U, S r is a Q by Q submatrix 
of S, and V r is a M by Q submatrix of V. Putting this in summation form, the resulting representation will be F vh = Zs(i) 
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*Uj*v |l where i is greater than or equal to 1 but less than or equal to Q. 

In this representation, the vector s(i) is a diagonal of the matrix S r , Uj is the i th column vector of U r , and v f is the i th 
column vector of V r . Each component is a separable filter similar to that described above with the exception of a gain 
factor s(i). In other words, any non-separable filter of length N by M can be represented as the weighted summation 

5 of Q separable N by M filters. Thus, to implement a non-separable filter using the weighted averages of several of the 
filters, the hardware becomes a conglomeration of Q separable filter modules, Q multipliers, and an adder. Such a 
hardware implementation is illustrated in Figure 3. 

The non-separable filter 1 of Figure 3 includes a plurality of separable filters 2, 4. 6, 8. Each of these separable 
filters includes a vertical filter and a horizontal fitter thus making the separable filter a separable two-dimensional filter. 

10 For example, the two-dimensional separable filter 2 includes a vertical filter 3 and a horizontal filter 5; filter 4 includes 
a vertical filter 7 and a horizontal filter 9; filter 6 includes a vertical filter 11 and horizontal filter 1 3; and filter 8 includes 
a vertical filter 15 and horizontal filter 17. 

It is noted that if the non-separable filter 1 is a 3-dimensional filter, the individual separable filters 2, 4, 6, and 8 
would each include 3 individual filters each filtering a separate dimension of the space. Such three dimensional filters 

is are valuable when filtering moving image data such as in a motion picture or video recording. 

The output of each separable filter is fed into a multiplier which multiplies the output of the separable two dimen- 
sional filter by a gain factor s(i). For example, the output from the two-dimensional separable filter 2 is fed into multiplier 
10 and multiplied by the gain factor s1 ; the output from filter 4 is fed into multiplier 12 and multiplied by the gain factor 
s2; the output from filter 6 is fed into multiplier 14 and multiplied by the gain factor s3; and the output from filter 8 is 

20 fed into multiplier. 16 and multiplied by the gain factor sQ. 

The gain factor can be a predetermined gain factor or a gain factor set by the fuzzy classifier described above. In 
the preferred embodiment of the present invention, the gain factor s(i) is set by the fuzzy classifier in accordance with 
the identified image type or class of the pixel being processed as described above. 

The outputs from each of the multipliers 10, 12, 14, 16 are fed into an adder 18 which sums all the outputs to 

25 produce a two-dimensionally filtered pixel of image data. It is noted : that the summation of all of the gain factors s(i) 
is equal to 1 to ensure that the overall gain of the non-separable filter 1 is equal to 1. It is further noted that each 
separable filter 2, 4, 6, 8 receives the same pixel of image data in parallel. Moreover, each separable two-dimensional 
filter 2, 4, 6, 8 is supplied with the coefficients needed to perform the two-dimensional filtering on that particular pixel. 
These coefficients may be predetermined or they may be selected from a set of different coefficients depending on the 

30 image type or class of the pixel of image data being processed. 

Figure 4 illustrates the relationship of a non-separable two-dimensional filter in an image processing scheme. As 
illustrated in Figure 4, video data is fed in parallel to a segmentation circuit 300 and buffers 306. The segmentation 
circuit 300 may either be a conventional image segmentation circuit or the fuzzy segmentation circuit discussed above. 
The segmentation circuit 300 determines the image type or class of the pixel of image data being processed. This 

35 image type or class information is fed into a weighting coefficients and gain factor generator 302 which generates the 
weighting coefficients and gain factors to be utilized by the non-separable two dimensional filter 304. 

The segmentation circuit 300 also generates other information with respect to the image type or class which are 
called effects to be utilized further downstream in the image processing system. The non-separable two-dimensional 
filter 304 utilizes the weighting coefficients and gain factors to properly filter the pixels of image data received from 

40 buffer 306 to produce a two-dimensionally filtered pixel of image data. 

When compared to a direct implementation of a separable N by M filter,' the implementation of the non-separable 
filter utilizing a plurality of separable filters makes the design of an N by M filter very modular. This modularity makes 
the redesign of a new filter much faster as only the individual separable filter components need to be redone. In addition, 
if the separable module can be easily configured for many different sizes, the non-separable module inherits the same 

*S configurability. 

Another major advantage of this type of implementation for a non-separable filter is that because of its modular 
nature, a filter can be implemented using the minimum amount of hardware. For example, assuming that an 11 x 15 
non-separable filter is desired, after performing a singular value decomposition on the 11x15 non-separable filter, one 
realizes that only two elements of the gain vector s(i) are non zero and thus the filter matrix for the 11x15 filter has a 

50 rank of 2. This also would be true if the gain vector s(i) had only 2 elements that were much larger than all the other 
elements of gain vector s(i). In such a case, the filter can be implemented in hardware using only 2 separable filter 
blocks, not 11 blocks, as would be required if the filter matrix were of full rank. 

In addition to being useful for implementing non-separable filters, the implementation of the present invention can 
be easily adapted to implement time-varying filters. The separable filters may not have the ability to time vary their 

55 coefficients hence the separable filters cannot be used to implement a time varying filter. However, using the concepts 
of the present invention, these filters can be used with some minimal additional hardware to create a time-varying non- 
separable filter. 

For example, the non-separable filter described above can be made time-varying by making the vector s(i) a 
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function of time and position. If the vector s(i) is not static, but instead is time-varying, the overall filter can be time- 
varying as well. If desired, the vector values can change on sample by sample or on a pixel by pixel basis. The time- 
varying effect can be attained by having only a few programmable gains; i.e., not ail the coefficients need to be time- 
varying to have good time-varying characteristic. Although there is not total Ireedom in choosing the filter's performance 

5 given a certain time and location, the implementation of the present invention provides several degrees of freedom 
that greatly enhance the functionality of the filters sufficient for many practical applications. 

For example, suppose that there are three different separable filters, one is a lowpass filter, one is a mid range 
(bandpass) filter, and one is a high pass filler. Furlhermore, it is desirable to implement a combination of these filters 
that varies over time. Additionally, at some given instance, the lowpass filter is the desired effect. In such a scenario, 

10 the weighting on the lowpass filter would be 1 with zero weighting on the other two filters; i.e., the s(i) vector would 
equal [1 0 0]. At another point in time, it may be necessary to block the lowpass filter and attenuate the mid range filter 
by a factor of 0.5 and amplify the high pass filter by 1 .2. This is accomplished by assigning the values of the s(i) vector 
to be equal to [0 0.5 1.2]. It is noted that a bypass filter would be included in these examples to ensure a gain of 1 .0 
as described above with respect to fuzzy processing. 

15 An application where this flexibility provides a needed functionality is in providing filtering as required when using 

fuzzy image segmentation or fuzzy image processing. In the fuzzy situation, there are different pre-defined classes, 
each associated with a desired filter. Moreover when utilizing fuzzy image segmentation, a pixel is not assigned one 
class but is assigned a membership at each of the pre-defined classes. Dependent on the segmentation class vector, 
the desired filter will be a weighted summation of each of the associated filters. 

20 More specifically, for example, in a situation utilizing a two class fuzzy classifier, the first class may require the 

object to be lowpass filtered and the second class requires the object to be high pass filtered. If the object is of the 
type of the first class, the lowpass filter is implemented; however, if the object of the second class, the high pass filter 
is implemented. 

After fuzzy image segmentation, the object is determined to have a membership of 0.7 in the first class and 0.3 in 

25 the second class. Such a determination will result in the decision to implement a filter that is the summation of 0.7 of 
the lowpass filter and 0.3 of the high pass filter. Such an implementation can be easily accomplished by setting the s 
(i) vector to be equal to [0.7 0.30]. If the next pixel is classified differently, its associated filter will be formed by changing 
the vector s(i) to produce the desired coherent filter. 

Typically, a two-dimensional filter is implemented in hardware as a fixed size filter such as an N by M filter. Many 

30 times, the filter is not self-timed, which means that its output will be larger than the original input since convolution 
changes the size of a fixed length input. This not a desirable characteristic for processing documents where there is 
a size constraint due to the size of the display media; i.e., screen size or paper size. To address this situation, a self- 
timed two-dimensional filter is proposed. 

A self-timed filter is one that does not require the external system to feed it additional information such as scanlines, 

35 pixels, or control signals before or after the input line or page has been processed to preserve the dimensions of the 
output page. Consequently, the number of scanlines and pixels per scanline in the output page is kept equal to the 
number of scanlines and pixels per scanline as the input page. The self-timed filter is not limited to a particular size, 
but can be programmed to any size less than a maximum N by M wherein N is the number of elements in the slowscan 
direction and M is the number of elements in the fastscan direction. 

^0 The self-timed two-dimensional filter of the present invention is accomplished using the separable implementation 

described above with respect to the non-separable two-dimensional filter. In this separable implementation, the input 
video undergoes filtering in the slowscan direction (vertical direction) followed by filtering in the fastscan direction 
(horizontal direction). Although the slowscan and fastscan filters may reside on the same chip, each filter acts inde- 
pendently of the other. 

45 In a Q by R implementation (Q is less than or equal to N and R is less than or equal to M), the filter is connected 

to Q-1 external first in, first out devices (FIFOs) that are used to buffer video for the slowscan direction. In addition, 
window pointers, (tags that are passed along with the input video and describes pixel characteristics) which must 
correspond to the center slowscan element, are buffered by (Q-1)/2 FIFOs. The final video output of the filter can be 
filtered video or unfiltered video wherein the unfiltered video is produced during the bypass mode. The bypass path 

so mode is chosen when there is not enough context to enable proper filtering such as around the perimeter of page. 

During the processing of the input page, the filter will not produce any output while the first (Q-1 )/2 scanlines are 
being input to the filter. However, on the scanline following the ((Q-1)/2) th scanline, the self-timed filter will begin to 
output video in the bypass mode. Due to a lack of context in the bypass mode, the center slowscan element, which is 
created by buffering the input video with (Q-1)/2 FIFOs, will exit the filter unchanged during the bypass mode. A Q by 

55 R filter requires Q lines of context before it can begin filtering. Consequently, the filter will remain in the bypass mode 
for (Q-1 )/2 scanlines until it begins the Qth scanline at which time it will begin filtering and will output filtered video. It 
will stop filtering video when the last scanline of the input page has been processed. 

Since the filter did not begin to output data until the scanline following the ((Q-1 )/2) m scanline, (Q-1 )/2 additional 
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scanlines have to be created at the end of an input page to preserve the total number of scanlines in the output page. 
To address this situation, the self-timed two-dimensional N by M filter creates the (Q-1)/2 additional scanlines once 
the input page has ended to preserve the total number of scanlines in the output page. 

In this implementation, a scan page is described by three signals; pagesync, linesync, and video valid. The first 

5 signal, pagesync, determines when a page is active. The second signal, linesync, determines when a scanline is active. 
The third signal is a video or pixel enable signal that indicates valid video during a scanline. The creation of (Q-1)/2 
additional scanlines once the input page has ended is done by creating a new set of internal signals that function as 
the page active, line active, and video enable signals for the rest of the filter. These signals also control the flushing 
of the last ((Q-1)/2) scanlines of video that are stored in the FIFO. 

to Figure 5 illustrates an example of a self-timed 9x15 two-dimensional fitter in accordance with the present invention. 

As illustrated in Figure 5, the self-timed 9 x 15 filter 500 receives the input signals linesync, video valid, pagesync, and 
system clock. These signals are buffered and then fed into a control logic circuit 502 which generates internal signals 
used for the self -timing mechanism. In the preferred embodiment of the present invention, the new internal signals are 
new linesync, new video valid, and new pagesync. 

15 These three new signals are fed into the non-separable two-dimensional filtering circuit 501 which is implemented 

in accordance with the non-separable filter 1 illustrated in Figure 3. The self-timed non-separable two-dimensional filter 
500 also receives nine scanlines ol video data wherein eight scanlines of video data are fed into FIFOs prior to entering 
the non-separable two-dimensional filtering circuit 501 . A delay circuit 503 is also included for delaying the video output 
and the classification signal. 

20 in this situation, while the first four scanlines of the input page are being input into the self-timed two-dimensional 

filter 500, the self-timed two-dimensional filter 500 will not output any video data. When the self-timed two-dimensional 
filter 500 begins to process the fifth scanline of the input page, the filter 500 will begin to output video data in the bypass 
mode and continue to process video in this mode until the ninth scanline of the input page is received. At this time, the 
self-timed two-dimensional filter 500 will actually begin the filtering process. 

25 The self-timed two-dimensional filter 500 will continue to filter video data until the last line of the active page has 

ended. At this time, the self-timed two-dimensional filter 500 will generate four additional linesync signals through 
internal signal new linesync, thereby flushing out the last four scanlines of video data from the FIFO buffers and out- 
putting this video in the bypass mode. 

To facilitate a better understanding of the self -timed two-dimensional filter as illustrated in Figure 5, a brief expla- 

30 nation of the timing aspect of this filter will be discussed below in relerence to the timing diagram illustrated in Figure 
6. More specifically, Figure 6 illustrates the input signals pagesync and linesync (A and B), the internal signals new 
pagesync and new linesync (C and D). and the output signals output pagesync and output linesync (E and F). Moreover, 
Figure 6 illustrates the timing diagram of a self-timed two-dimensional filter having dimensions of 9 by 1 5. 

As illustrated in Figure 6, when the pagesync signal (A) becomes active, the self-timed two-dimensional filter 

35 causes the internal new pagesync signal (C) to also become active. Moreover, when the self-timed two-dimensional 
filter receives an active linesync signal (B), the self-timed two-dimensional filter causes the internal new linesync signal 
(D) to also become active. However, in this example, the output pagesync signal (E) and the output linesync signal (F) 
are not caused to be active until a later time. 

More specifically, the output pagesync signal (E) does not become active until the fifth input linesync signal (B) is 

40 received as indicated by dotted line 600. At this point, the output pagesync signal (E) becomes active and the first 
output linesync signal (F) becomes active. This point in time also corresponds to the beginning of the first of the bypass 
mode wherein the self -timed two-dimensional filter outputs unfiltered video data since the filter has not received enough 
scanlines of video data to properly begin filtering. This bypass mode continues until the ninth input linesync signal (B) 
is received wherein the self-timed two-dimensional filter switches to the filtering mode and outputs properly tittered 

45 image. data. 

At some later time, indicated by dotted line 601 , when the received pagesync signal (A) becomes inactive indicating 
that the input page has ended, the self-timed two-dimensional filter causes the internal new pagesync signal (C) to 
remain active for an additional four scanline cycles (as shown by 602 (signal (C)) and 603 (signal (D))) and so as to 
flush out the buffers. Upon receiving an inactive pagesync signal, the self-timed two-dimensional filter switches from 
so the filtering mode back to the bypass mode so as to flush out the FIFOs with unfiltered video data. In the output linesync 
signal (F), the initial four scanlines are indicated by 604, the last four (flushing) scanlines are indicated by 606, and 
the scanlines between are indicated by 605. The scanlines indicated by 606 correspond to those in new linesync 
indicated by 603. 

Although the two-dimensional filters of the present invention has been described with respect to a single filter 
55 architecture, the two-dimensional filters of the present invention can also be implemented as a plurality ol cascadable 
filters. In other words, a large two-dimensional filter can be built by using many smaller cascadable filters. These 
cascadable filters are separable and symmetric and may be of any size N by M wherein N is the number of slowscan 
elements and M is the number of fastscan elements. 
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The cascadablo filter has an extra cascade input and cascade output that are only used when multiple filters are 
cascaded together. Thus, any number of cascadable filters can be cascaded together to provide the filter size. For 
example, when two (N by M) filters are cascaded together, the maximum resulting filter can be an (2N-1) by (2M-1) 
filter. Moreover, when three (N by M) filters are cascaded together, the maximum resulting filter can be (3N-1) by (3M- 
1 ) filter, etc. Also, when two N filters are cascaded together and combined with a M filter, a (2N-1 ) by M filter is created. 

To explain the cascadable concept more clearly, the implementation of a 7 x 1 5 filter will be utilized as an example. 
Moreover, in this example, the individual components of the two-dimensional filter (the slowscan filter and the fastscan 
filter) will be examined individually 

As noted above, Figure 7 illustrates a cascadable seven element slowscan filter. This cascadable slowscan filter 
receives pixels of image data from seven different scanlines wherein a pair of pixels of image data are fed to each 
adder 701, 702, 703, and 704. It is noted that since Figure 7 illustrates a seven element slowscan cascadable filter, 
the input X to adder 704 is tied to zero. 

The output from each adder is fed into a separate multiplier 705, 706, 707, and 708 wherein the individual sums 
are multiplied by the weighting coefficients A. B, C, D respectively. The weighted results from the multipliers are then 
fed into adder 709 which produces a seven element slowscan filtered pixel of image data. The output from adder 709 
is fed to adder 710 which sums the seven element slowscan filtered pixel of image data with the filtered image data 
from another cascadable slowscan filter as indicated by input 711 . This input can be tied to zero if it is not desirable to 
make a fitter greater than seven elements, otherwise input 711 corresponds to the output from a previous cascade 
filter (not shown). Output 712 is used when there is a following or subsequent cascade filter (not shown), and output 
71 3 is used when there are no subsequent cascade filters. 

Suppose that it is desirable to cascade two cascadable slowscan filters to create one that is fifteen elements wide. 
More specifically, it is desirable to cascade two slowscan filters which will generate image data that is represented by 
a value that is equal to A(line 0 + line 14) + B(line 1 + line 13) + C(line 2 + line 12) + D(line 3 + line 11) + E(line 4+ line 
10) + F(line 5 + line 9) + G(line 6 + line 8) + H(line 7). 

Conventionally, such a fifteen element slowscan filter would be implemented as illustrated in Figure 8. In other 
words, each of the eight operations described above to generate the proper value for the pixel ol image data would be 
implemented in parallel and fed to a single adder 416. Adders 401 , 403, 405, 407, 409, 411 , 413 add respective line 
pairs (0+14), 1+13), (2+12), (3+11), (4+10), (5+9) and (6+8). Each output is then passed to a respective multiplier 402, 
404, 406, 408, 410, 412, 414 where it is multiplied by a predetermined coefficient as shown. Line 7 is input directly to 
multiplier 415 and multiplied by its associated coefficient. The output from each multiplier is then added in adder 416 
to provide an output 418. Each of the coefficients are fixed but they may be determined using fuzzy classification. 
However, as noted above, such a large filter can be implemented utilizing a plurality of cascadable filters. 

More specifically, as illustrated in Figure 9, the fifteen element slowscan filter can be implemented utilizing two 
smaller cascadable filters 800 and 900. In this example, each cascadable filter is similar to the cascadable filter as 
discussed above with respect to Figure 7. Adders 801, 803, 805, 807, 809, 810 and multipliers 802 : 804, 806, 808 
correspond to respective ones of adders 701 . 703, 705, 707, 709, 710 and multipliers 702, 704, 706, 708 of Figure 7. 
However in this case, the cascadable filter 900 receives the cascadable filtered image data output 812 from cascadable 
filter 800 as input 91 2 into adder 91 0. Adder 91 0 adds together the cascaded filtered image data from cascadable filter 
800 with the filtered image data produced by the various elements of cascadable filter 900 to produce a final filtered 
result which represents a pixel of image data which has been filtered by a fifteen element slowscan filter. In a similar 
way to the comparison made between Figure 7 and filter 800, filter 900 has identical corresponding components 901, 
902, 903, 904, 905, 906, 907, 908, 909, 910 for processing the inputs not processed by filter 800. 

In other words, the output from adder 910 generates a value which is represented by the equation given above. 
Moreover, the inclusion of an additional cascadable filter as illustrated in Figure 7, can result in a twenty-three element 
slowscan filter (not shown). Moreover, the inclusion of two more cascadable filters, as illustrated in Figure 7, can result 
in a thirty-one element slowscan filter (not shown). 

Not only can cascadable filters be applied to the slowscan direction, it is also applicable to the fastscan direction. 
Figure 10 illustrates a fastscan cascadable filter wherein input 321 represents the cascadable output from another 
fastscan cascadable filter (not shown). If one desires that the present cascadable filter, as illustrated in Figure 10, to 
remain a fifteen element fastscan filter, the input 321 is tied to zero and the input Y to adder 315 is also tied to zero. 
As before, pairs of inputs are input to respective ones of adders 301, 303, 305, 307, 309, 311, 313 and the outputs 
therefrom pass to respective multipliers 302, 304, 306, 308, 310, 312, 314 where they are multiplied by suitable coef- 
ficients. The outputs from these multipliers together with those from multiplier 316 (associated with adder 315) and 
multiplier 31 7 are passed to adder 31 8 where they are summed. The output from adder 31 8 passes to adder 31 9 where 
any cascaded output is added (input 321) to provide output 322. 

If it is desired to implement a thirty-one element wide fastscan filter, one would cascade together two fastscan 
filters as illustrated in Figure 10 wherein the output 320 of the first cascadable filter would be input to the input terminal 
that is equivalent to the input terminal 321 in the second cascadable fastscan filter. In this way, two smaller fastscan 
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filters can be cascaded to create a larger fastscan filter. 

An example of a chip having a cascadable filtering architecture is illustrated in Figure 21 . The filtering chip 2000 
includes N cascadable filters 2001 , 2005, 2011. 2015. 2017. The filtering chip 2000 also includes N multiplexers 2003, 
2007, 2013, 2019 wherein a single multiplexer is connected between two cascadable filters. The multiplexers receive 
5 the cascadable filtered image data from the downstream filter (i.e., multiplexer 2003 receives the cascadable filtered 
image data from filterl 2001) and a zero value at the multiplexers two input terminals. 

If the output of the downstream filter is to be cascaded into the next filter, a logically enabling signal is received by 
the multiplexer so that the filtered data is input into the upstream filter. For example, il filterl 2001 and filler2 2005 where 
to be cascaded together, multiplexer 2003 would receive a logically enabling signal, causing multiplexer 2003 to pass 
io the filtered output of filterl 2001 to filter2 2005. If the two filters were not cascaded together, the multiplexer 2003 would 
receive a logically non-enabling signal, causing multiplexer 2003 to pass on a zero value to filter2 2005, thereby pre- 
venting the cascading of the two filters. 

Although tho present invention has been described in detail above, various modifications can also be implemented. 
For example, the preferred embodiment of the present invention has been described with respect to a xerographic 
is printing system; however, these fuzzy methods and filters are readily implemented in a thermal inkjet system, a display 
system, or other image processing system. 

Moreover, the image processing system of the present invention can be readily implemented on a general purpose 
computer, a personal computer or workstation. The image processing system of the present invention can also be 
readily implemented on an ASIC, thereby enabling the placement of this process in a scanner, electronic subsystem, 
20 printer, or display device. 

The present invention has been described with respect to a video range of 0 to 255. However, it is contemplated 
by the present invention that the video range can be any suitable range to describe the grey level of the pixel being 
processed. Furthermore, the present invention is readily applicable to any image processing system, not necessarily 
a binary output device. It is contemplated that the concepts of the present invention are readily applicable to a four- 
25 level output terminal or higher. 

Also, the present invention has been described, with respect to the fuzzy classification and fuzzy processing rou- 
tines, that the scalar values are determined using the weighted sum of the centroid method since the centroids in the 
preferred embodiment are non-overlapping (the classes are non-overlapping). However, the present invention is readily 
applicable to a system with overlapping classes. Such an extension is readily known to those skilled in the art of fuzzy 
30 logic. 

Lastly, the present invention has been described with respect to a monochrome or black/white environment. How- 
ever, the concepts of the present invention are readily applicable to a color environment. Namely, the image processing 
operations of the present invention can be applied to each color space value representing the color pixel. 

35 

Claims 

1. A filter device providing a plurality of two-dimensional digital filters, comprising: 

40 a plurality of one-dimensional first stage filters for producing a one-dimensional filtered output signal from an 

array ol input pixel signals; 

a plurality of one-dimensional second stage filters, each one-dimensional second stage filter receiving output 
signals from a selected first stage filter and applying a second filtering operation thereto, to produce two- 
dimensional filtered output signals; and 
45 a weighting circuit to modify each of the produced two-dimensional filtered output signals according to weight- 

ing coefficients. 

2. A filter device as claimed in Claim 1 , further comprising: 

an adder to combine all the modified filtered signals to produce a modified digital image signal. 

so 

3. A filter device as claimed in Claim 1 or 2, further comprising: 

a segmentation circuit to determine a type of class of the digital image data being processed by the image 
processing device and to produce the weighting coefficients for the weighting circuit. 

55 4. A filter device as claimed in any one of Claims 1 to 3, further comprising: 

a timing circuit to provide self-timing for said plurality of one dimensional first dimension and second dimen- 
sion filters. 
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5. A filter device as claimed in any one of Claims 1 to 4, wherein said plurality of one-dimensional first dimension 
filters are vertical filters. 

6. A filter device as claimed in any one of Claims 1 to 5, wherein said plurality of one-dimensional first dimension 
filters are slow scan filters. 

7. A filter device as claimed in any one of Claims 1 to 6, wherein said plurality of one-dimensional second dimension 
Niters are horizontal filters. 

8. A filter device as claimed in any one of Claims 1 to 7, wherein said plurality of one-dimensional second dimension 
filters are fast scan filters. 

9. A method for implementing a plurality of two-dimensional digital filters, comprising the steps of: 

(a) applying a first set of a plurality of one-dimensional first dimension filter to digital signals oriented along a 
first direction, each first dimension filters operating on the digital signals in accordance with a set of filter 
coefficients; 

(b) applying a second set of a plurality of one-dimensional second dimension filters to the signals filtered by 
said step (a) oriented along a second direction in accordance with a set of filter coefficients; and 

. (c) weighting each of the signals produced by said step (b) according to a set of weighting coefficients. 

10. A method as claimed in Claim 9, further comprising the step of adding together all the weighted signals produced 
by said step (c) to produce a modified digital signal. 

11. A method as claimed in Claim 9 or 10, further comprising the steps of analyzing the digital signals to determine a 
type of class associated with the digital signals; and producing the set of weighting coefficients based on the 
determined type of class. 
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